Observational studies have recently received significant attention from the machine learning community due to the increasingly available non-experimental observational data and the limitations of the experimental studies, such as considerable cost, impracticality, small and less representative sample sizes, etc. In observational studies, de-confounding is a fundamental problem of individualised treatment effects (ITE) estimation. This paper proposes disentangled representations with adversarial training to selectively balance the confounders in the binary treatment setting for the ITE estimation. The adversarial training of treatment policy selectively encourages treatment-agnostic balanced representations for the confounders and helps to estimate the ITE in the observational studies via counterfactual inference. Empirical results on synthetic and real-world datasets, with varying degrees of confounding, prove that our proposed approach improves the state-of-the-art methods in achieving lower error in the ITE estimation.

在生物医学语料库中预先培训的语言模型，例如Biobert，最近在下游生物医学任务上显示出令人鼓舞的结果。另一方面，由于嵌入尺寸，隐藏尺寸和层数等因素，许多现有的预训练模型在资源密集型和计算上都是沉重的。自然语言处理（NLP）社区已经制定了许多策略来压缩这些模型，利用修剪，定量和知识蒸馏等技术，从而导致模型更快，更小，随后更易于使用。同样，在本文中，我们介绍了六种轻型模型，即Biodistilbert，Biotinybert，BioMobilebert，Distilbiobert，Tinybiobert和Cmpactactbiobert，并通过掩护的语言在PubMed DataSet上通过掩护数据进行了知识蒸馏而获得的知识蒸馏来获得。建模（MLM）目标。我们在三个生物医学任务上评估了所有模型，并将它们与Biobert-V1.1进行比较，以创建有效的轻量级模型，以与较大的对应物相同。所有模型将在我们的HuggingFace配置文件上公开可用，网址为https://huggingface.co/nlpie，用于运行实验的代码将在https://github.com/nlpie-research/compact-compact-biomedical-transformers上获得。

在电子健康记录（EHRS）中，不规则的时间序列（ITS）自然发生，这是由于患者健康动态而自然发生，这是由于医院不规则的探访，疾病/状况以及每次访问时测量不同生命迹象的必要性。其目前的培训挑战机器学习算法主要建立在相干固定尺寸特征空间的假设上。在本文中，我们提出了一种新型的连续患者状态感知器模型，称为铜，以应对其在EHR中。铜使用感知器模型和神经普通微分方程（ODE）的概念来学习患者状态的连续时间动态，即输入空间的连续性和输出空间的连续性。神经ODES可以帮助铜生成常规的时间序列，以进食感知器模型，该模型具有处理多模式大规模输入的能力。为了评估所提出的模型的性能，我们在模仿III数据集上使用院内死亡率预测任务，并仔细设计实验来研究不规则性。将结果与证明所提出模型的功效的基准进行了比较。

疾病进展模型的一个特殊挑战是疾病的异质性及其在患者中的表现。现有方法通常假设存在单一疾病进展特征，这对于诸如帕金森氏病等神经退行性疾病不太可能。在本文中，我们提出了一个分层的时间序列模型，该模型可以发现多种疾病进展动力学。提出的模型是投入输出隐藏模型的扩展，该模型考虑了患者健康状况和处方药的临床评估。我们使用合成生成的数据集和用于帕金森氏病的现实世界纵向数据集说明了模型的好处。

为了训练强大的深神经网络（DNNS），我们系统地研究了几种目标修饰方法，其中包括输出正则化，自我和非自动标签校正（LC）。发现了三个关键问题：（1）自我LC是最吸引人的，因为它利用了自己的知识，不需要额外的模型。但是，在文献中，如何自动确定学习者的信任程度并没有很好地回答。 （2）一些方法会受到惩罚，而另一些方法奖励低渗透预测，促使我们询问哪一种更好。 （3）使用标准训练设置，当存在严重的噪音时，受过训练的网络的信心较低，因此很难利用其高渗透自我知识。为了解决问题（1），采取两个良好接受的命题 - 深度神经网络在拟合噪声和最小熵正则原理之前学习有意义的模式 - 我们提出了一种名为Proselflc的新颖的端到端方法，该方法是根据根据学习时间和熵。具体而言，给定数据点，如果对模型进行了足够的时间训练，并且预测的熵较低（置信度很高），则我们逐渐增加对预测标签分布的信任与其注释的信任。根据ProSelfLC的说法，对于（2），我们从经验上证明，最好重新定义有意义的低渗透状态并优化学习者对其进行优化。这是防御熵最小化的防御。为了解决该问题（3），我们在利用低温以纠正标签之前使用低温降低了自我知识的熵，因此修订后的标签重新定义了低渗透目标状态。我们通过在清洁和嘈杂的环境以及图像和蛋白质数据集中进行广泛的实验来证明ProSelfLC的有效性。此外，我们的源代码可在https://github.com/xinshaoamoswang/proselflc-at上获得。

变形金刚是一个有前途的神经网络学习者，在各种机器学习任务中取得了巨大的成功。由于最近的多模式应用程序和大数据的流行率，基于变压器的多模式学习已成为AI研究中的热门话题。本文对以多模式数据为导向的变压器技术进行了全面调查。该调查的主要内容包括：（1）多模式学习，变压器生态系统和多模式大数据时代的背景，（2）从几何拓扑的观点，从几何拓扑的角度来看，对香草变压器，视觉变压器和多模式变压器的理论回顾， （3）通过两个重要范式（即用于多模式预处理和特定的多模式任务）对多模式变压器应用的综述，（4）摘要多模式变压器模型和应用共享的共同挑战和设计，以及（5）讨论社区的开放问题和潜在的研究方向。

Covid-19的早期检测是一个持续的研究领域，可以帮助潜在患者的潜在患者进行分类，监测和一般健康评估，并可能降低应对冠状病毒大流行病的医院的运营压力。在文献中使用了不同的机器学习技术，用于使用常规临床数据（血液测试和生命体征）来检测冠状病毒。使用这些型号时，数据漏洞和信息泄漏可以带来声誉损害并导致医院的法律问题。尽管如此，保护避免潜在敏感信息泄漏的医疗保健模型是一个被人吸引人的研究区。在这项工作中，我们检查了两种机器学习方法，旨在预测使用常规收集和易于使用的临床数据的患者的Covid-19状态。我们雇用对抗性培训来探索强大的深度学习架构，保护与有关患者的人口统计信息相关的属性。我们在这项工作中检查的两种模型旨在保持对抗对抗攻击和信息泄漏的敏感信息。在一系列使用来自牛津大学医院的数据集，Bedfordshire医院NHS Foundation Trust，大学医院伯明翰NHS基金会信托，而朴茨茅斯医院大学NHS信任我们训练并测试两个神经网络，以使用来自基本实验室血液的信息预测PCR测试结果的神经网络对患者到达医院的测试和生命体征。我们评估其每个模型的隐私水平可以提供和展示我们提出的架构对可比基线的效力和稳健性。我们的主要贡献之一是，我们专门针对具有内置机制的有效Covid-19检测模型的开发，以便选择性地保护对抗对抗攻击的敏感属性。

In this paper, we propose a novel technique, namely INVALIDATOR, to automatically assess the correctness of APR-generated patches via semantic and syntactic reasoning. INVALIDATOR reasons about program semantic via program invariants while it also captures program syntax via language semantic learned from large code corpus using the pre-trained language model. Given a buggy program and the developer-patched program, INVALIDATOR infers likely invariants on both programs. Then, INVALIDATOR determines that a APR-generated patch overfits if: (1) it violates correct specifications or (2) maintains errors behaviors of the original buggy program. In case our approach fails to determine an overfitting patch based on invariants, INVALIDATOR utilizes a trained model from labeled patches to assess patch correctness based on program syntax. The benefit of INVALIDATOR is three-fold. First, INVALIDATOR is able to leverage both semantic and syntactic reasoning to enhance its discriminant capability. Second, INVALIDATOR does not require new test cases to be generated but instead only relies on the current test suite and uses invariant inference to generalize the behaviors of a program. Third, INVALIDATOR is fully automated. We have conducted our experiments on a dataset of 885 patches generated on real-world programs in Defects4J. Experiment results show that INVALIDATOR correctly classified 79% overfitting patches, accounting for 23% more overfitting patches being detected by the best baseline. INVALIDATOR also substantially outperforms the best baselines by 14% and 19% in terms of Accuracy and F-Measure, respectively.

The recent increase in public and academic interest in preserving biodiversity has led to the growth of the field of conservation technology. This field involves designing and constructing tools that utilize technology to aid in the conservation of wildlife. In this article, we will use case studies to demonstrate the importance of designing conservation tools with human-wildlife interaction in mind and provide a framework for creating successful tools. These case studies include a range of complexities, from simple cat collars to machine learning and game theory methodologies. Our goal is to introduce and inform current and future researchers in the field of conservation technology and provide references for educating the next generation of conservation technologists. Conservation technology not only has the potential to benefit biodiversity but also has broader impacts on fields such as sustainability and environmental protection. By using innovative technologies to address conservation challenges, we can find more effective and efficient solutions to protect and preserve our planet's resources.

Variational autoencoders model high-dimensional data by positing low-dimensional latent variables that are mapped through a flexible distribution parametrized by a neural network. Unfortunately, variational autoencoders often suffer from posterior collapse: the posterior of the latent variables is equal to its prior, rendering the variational autoencoder useless as a means to produce meaningful representations. Existing approaches to posterior collapse often attribute it to the use of neural networks or optimization issues due to variational approximation. In this paper, we consider posterior collapse as a problem of latent variable non-identifiability. We prove that the posterior collapses if and only if the latent variables are non-identifiable in the generative model. This fact implies that posterior collapse is not a phenomenon specific to the use of flexible distributions or approximate inference. Rather, it can occur in classical probabilistic models even with exact inference, which we also demonstrate. Based on these results, we propose a class of latent-identifiable variational autoencoders, deep generative models which enforce identifiability without sacrificing flexibility. This model class resolves the problem of latent variable non-identifiability by leveraging bijective Brenier maps and parameterizing them with input convex neural networks, without special variational inference objectives or optimization tricks. Across synthetic and real datasets, latent-identifiable variational autoencoders outperform existing methods in mitigating posterior collapse and providing meaningful representations of the data.